Serial Attached SCSI (Small Computer System Interface), or SAS, is a connection-oriented protocol that allows storage devices, like servers and disk drives, to communicate through a network of high-speed serial physical interconnections. Intermediate devices called expanders manage connections between a host device and a target drive. SAS expanders act as connection management agents, much like a switch element, having physical connections to multiple host devices or disk drives simultaneously. In SAS, a physical link is typically a set of four signal lines used as two differential pairs. A “PHY” is a transceiver that electrically interfaces with a physical link combined with the portions of the protocol that encode data and manage the reset sequences. A port is created when one or more PHYs share the same address and attach to a device through one or more physical links.
A connection is a temporary association between a SAS initiator port and a SAS target port. During a connection all Dwords (for “double word”—two words each composed of two bytes) from the SAS initiator port are forwarded to the SAS target port, and all Dwords from the SAS target port are forwarded to the SAS initiator port. A connection is established through a connection signaling process defined in the SAS standard. Though the SAS data transfer protocol is connection-oriented, the connection signaling protocol is packet-oriented.
There are two types of SAS frames: those transmitted when there is no connection established, and those transmitted during a connection. SAS frames transmitted when there is no connection established are called SAS address frames. The two address frames defined in the SAS link layer are the IDENTIFY address frame and the OPEN address frame. Transmission of these frames is immediately preceded by an SOAF (start of address frame) primitive and immediately followed by an EOAF (end of address frame) primitive. To establish a connection, a source port (in either an initiator or a target) sends a connection request in the form of an “open address frame” data packet. The expander device processes the open address frame and forwards the modified open address frame to the next expander device on the pathway or to the final destination port (in either an initiator or a target).
FIG. 1 shows an example of how a connection is setup normally in a SAS network between an initiator and a target. As shown in step 102 of FIG. 1, a host bus adaptor (HBA) 120 sends a connection request, or an open address frame, to expander 122. After receiving the open address frame, the expander 122 determines the destination port based on the destination address in the open address frame and establishes an internal connection between the source port, at which the open address frame is received, and the destination port. The expander 122 then forwards the open address frame through the internal connection to the destination port, as illustrated in step 104. The expanders 124 and 126 then repeat what is done in expander 122 as shown in steps 106 and 108 in FIG. 1. After these relays, the open address frame finally reaches the final destination port in a hard disk drive (HDD) 128. The HDD 128 then responds to the open address frame by sending out an OPEN_ACCEPT primitive, see the following document for details on primitive definition: ANSI INCITS. T10/1601-D Revision 6—Serial Attached SCSI (SAS), Oct. 2, 2004 (also referred to as the SAS standard), which is incorporated herein by reference. The OPEN_ACCEPT primitive is then transferred along the physical connection all the way back to the original source port in the HBA 120, which completes the connection setup, as shown in step 110 of FIG. 1.
In expander devices, the SAS standard defines the following functions: two or more PHYs, expander connection manager (ECM) and expander connection router (ECR). An external expander port contains one or more physical PHYs. Each expander PHY contains an expander link layer with a connection state machine and a reset state machine. The expander link layers within an expander PHY request and respond to connection requests independently. A source PHY is defined as a PHY from which an open frame is received, and a destination PHY is defined as a PHY to which the source PHY is connected.
The expander connection manager (ECM) performs the following functions: a) maps a destination SAS address in a connection request to a destination PHY using direct, subtractive, or table routed addressing methods; b) arbitrates and assigns or denies path resources for connection requests following SAS arbitration and pathway recovery rules; and c) configures the ECR. The expander connection router (ECR) routes messages between pairs of expander phys (source PHY and destination) as configured by the ECM.
The SAS standard defines an open address frame that contains an identification of link protocol, connection type, destination port address, source port address, priority counters and CRC. Table 1 provides details on the contents of an open address frame, shown in relation to Dword number.
TABLE 1Open Address Frame ContentsDwordNumberDescriptionProcessing in expander1Protocol/Connection RateRouting: Arbitration2Destination SAS AddressRouting: Lookup[63:32]3Destination SAS Address[31:0]4Source SAS AddressRouting: Arbitration[63:32]5Source SAS Address [31:0]6Priority CounterRouting: Arbitration7Reserved8CRCError Checking
The 2nd and 3rd Dwords include destination SAS device address information. The destination SAS device address determines which PHY in the expander is the destination PHY to which the source PHY will be connected.
The 1st Dword contains an identification of the link protocol of the connection and connection rate. The 4th and 5th Dwords are the source port address and the 6th Dword comprises priority counters including Pathway Blocked Counter and Arbitration Wait Timer. The parameters of connection link protocol, connection rate, source port address, Pathway Blocked Counter and Arbitration Wait Timer are all used for arbitration when multiple connection requests to the same destination PHY are pending in an expander. The last Dword is the CRC of the whole open address frame and is used for checking frame error. If a CRC error is detected, the open request is discarded.
The SAS standard also defines the process of routing an open address frame in source PHY, ECM and ECR. FIG. 2 illustrates an example of how the process can be practically implemented. Reference numerals 201-209 in FIG. 2 indicate an event number of a processing event; an event with a larger number depends on an event with a lower number. Note that only one source PHY and destination PHY related to the open address frame are shown here.
The event sequence illustrated in FIG. 2 is as follows:                Step 201. Receive and store open address frame (in source PHY 220). When an open address frame arrives at a PHY, it will be first stored in an open address frame context in the PHY.        Step 202. CRC check (in source PHY). After the whole open address frame is received, the CRC will be checked. If the CRC is correct, go to the next step. If CRC check fails, discard the open address frame.        Step 203. Open request to ECM (in source PHY). An open request is asserted to ECM 224 to indicate there is an open request pending in the PHY.        Step 204. Enable load (in ECM). After receiving an open request, the ECM determines whether to load the open request context. Open request context is defined as a subset of the open address frame comprising routing information such as SAS destination, source addresses and priority for address lookup and arbitration in ECM. An open request context can be loaded only if the previous arbitration window is finished.        Step 205. Load open request context (in source PHY) and store open request context (in ECM). After receiving load enable confirmation from ECM, the source PHY starts to load the open request context to the ECM, and the ECM stores it in ECM open request context registers.        Step 206. Address lookup (in ECM). The ECM starts an address lookup to determine which PHY would be a destination PHY after the whole open frame context is received from the source PHY.        Step 207. Arbitration (in ECM). If more than one open request is designated to one destination PHY, the ECM will perform arbitration based on the priorities of the open request indicated in the open request context.        Step 208. PHY selection (in ECM). When arbitration is finished, the ECM acknowledges to the source PHY whether the open request is accepted, rejected or is pending. In the meantime, if the open request is accepted, the ECM will configure ECR 226 to setup a connection between the source PHY and the destination PHY 228.        Step 209. Forward open address frame. After the arbitration result is received from the ECM, if it is to accept the open request, the source PHY start to forward the open address frame with modified priority counters through the ECR to the destination PHY along the connection setup by ECM in step 208.        
Note that parallel to these steps, the arbitration wait timer keeps ticking in the source PHY as long as the open address frame has been received. As a result, the final forwarded open address frame is different from what is received in the source PHY, since the arbitration wait timer value is set to be the current value of the timer, not the initial received value.
For a SAS expander device, the connection setup time delay between receiving an open address frame in the source PHY and transmitting the open address frame in the destination PHY is crucial for entire SAS network performance in setting up connections. The intuitive way of implementing open address frame processing in both source PHY and ECM as described above uses the store-and-forward operation, which serializes each step of the processing. This negates any advantages offered by the parallelization of the steps. There are times during operation of such a device when transmission of a packet could begin before the entire packet had been received, and the packet could be passed straight through from one network to another, with practically no delay. This mode of operation is referred to as cut-through operation. Some bridges provide for operation in this mode, in a packet-switched networking context. Some examples include: U.S. Pat. No. 5,307,345 issued Apr. 1, 1994 to Philip P. Lozowick and Siman-Tov Ben-Michael; U.S. Pat. No. 6,522,656 issued Feb. 18, 2003 to Curtis D. Gridley; U.S. Pat. No. 6,233,243 issued May 15, 2001 to Mahesh N Ganmukhi and Prasasth R. Palnati; U.S. Pat. No. 6,728,249 issued Apr. 27, 2004 to Chi-Hua Chang; and Serial Attached SCSI (SAS) Standard (ANSI INCITS. T10/1601-D Revision 6—Serial Attached SCSI (SAS), Oct. 2, 2004). However, there are no similar approaches for SAS expander devices.
It is, therefore, desirable to provide a system and method for handling SAS open frame and expander device operation including at least some steps in parallel.